Learning Comprehensive Motion Representation for Action Recognition

For action recognition learning, 2D CNN-based methods are efficient but may yield redundant features due to applying the same convolution kernel each frame. Recent efforts attempt capture motion information by establishing inter-frame connections while still suffering limited temporal receptive field or high latency. Moreover, feature enhancement is often only performed channel space dimension in recognition. To address these issues, we first devise a Channel-wise Motion Enhancement (CME) module adaptively emphasize channels related dynamic with channel-wise gate vector. The gates generated CME incorporate from all other frames video. We further propose Spatial-wise (SME) focus on regions critical target motion, according point-to-point similarity between adjacent maps. intuition that change of background typically slower than area. Both and SME have clear physical meaning capturing clues. By integrating two modules into off-the-shelf network, finally obtain Comprehensive Representation (CMR) learning method for recognition, which achieves competitive performance Something-Something V1 & V2 Kinetics-400. On reasoning datasets V2, our outperforms current state-of-the-art 2.3% 1.9% when using 16 as input, respectively.